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"MEDICAL VIEWING SYSTEM AND METHOD FOR DETECTING AND ENHANCING 
BOUNDARY SIRUCTURES IN NOISY IMAGES" 



The invention relates to a medical viewing system for displaying a sequence of 
images of a medical intervention that comprises moving and/or positioning a tool with 
respect to a body organ. The invention also relates to a computer executable image 
10 processing method to be used in said system. The invention further relates to a medical 
examination apparatus coupled to such a system. The invention finds for example its 
application in the medical field of cardiology, for registering and enhancing thin objects of 
interest such as vessel walls in arteriograms. 



interventions under x-ray fluoroscopy is already known firom the publication entitled "2-D 
Guide wire tracking during endovascular interventions" by Shirley A. M. Baert, in "Medical 
Image Computing and Computer-Assisted Intervention" -MICCAI 2000, S. L. Delp, A. M. 
DiGioia, B. Jaramaz (eds), vol. 1935 of Lecture Notes in Computer Science, Springer- Verlag, 

20 Berlin, 2000, pp. 727-734. This publication describes a method that can be used in low 

quality fluoroscopic images to estimate tibie position of the guide wire in world coordinates. A 
two-step procedure is utilized to track the guide-wire in subsequent firames. In a first step, a 
rough estimate of the displacements of the guide wire is obtained using a template matching 
procedure on a spline model. In a second step, the position of the guide-wire is optimized, by 

25 fitting the guide-wire to a feature image in which line-like structures are enhanced. In this 
optimization step, the influence of the scale at which the feature is calculated and the 
additional value of using directional information is investigated. The method is applied both 
on the original and subtraction images. Using proper parameter settings, the guide-wire could 
be successfiilly tracked based on the original images, in 141 out of 146 firames firom 5 image 

30 sequ^ces. 

A method for extracting stilts in medical images is already known firom the 
publication entitled "Deformable Boimdary Detection of Stents in Angiographic Images", by 
loannis Kompatsiaris et alii, in IEEE TRANSACTIONS ON MEDICAL IMAGING, 
VOL.19, No. 6, JUNE 2000, pages 652-662. This document describes an image processing 
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Background of the Invention 
A method to extract and track the position of a guide wire during endovascular 
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method for deformable boundary detection of medical tools, called stents, in angiographic 
images. A stent is a surgical stainless steel coil that is placed in the artery in order to improve 
blood circulation in regions where a stenosis has appeared. Assuming initially a set of three- 
dimensional (3-D) models of stents and using perspective projection of various deformations 
5 of the 3-D model of the stent, a large set of two-dimensional (2-D) images of stents is 
constructed. These synthetic images are then used as a training set for deriving a multi- 
variate density estimate based on eigenspace decomposition and formulating a maximum- 
likelihood estimation firamework in order to reach an initial rough estimate for automatic 
object recognition. Then, the silhouette of the detected stent is refined using a 2-D active 

10 contour (snake) algorithm, integrated with an iterative initialization technique, which takes 
into consideration the geometry of the stent. As disclosed in the cited publication, when a 
narrowing called stenosis is identified in a coronary artery of a patient, a procedure called 
angioplasty may be prescribed to improve blood flow to the heart muscle by opening the 
blockage. In recent years, angioplasty increasingly employs a stent implantation technique. 

15 This stent implantation technique includes an operation of stent placement at the location of 
the detected stenosis in order to efficiently hold open the diseased vessel, as illustrated by 
figure 2 of the cited publication. Stent placemrat helps many patients to avoid emergency 
heart bypass surgery and/or heart attack (myocardial infiraction). The stent, as illustrated by 
figure 1 of the cited publication, is a small, slotted, stainless steel tube cut by a precision laser 

20 for forming a coil. It is wrapped tightly around a balloon attached to a monorail introduced 
by way of a catiieter and a guide-wire forming a device called balloon-tipped catheter. This 
balloon-tipped catheter is introduced into the artery through a small incision. Once in place, 
the balloon is inflated in order to expand the coil. Once expanded, the stent, which can be 
considered as a permanent implant, acts like a scaffold keeping the artery wall open. This 

25 allows more blood flow to the heart muscle. 

Summary of the Invention 
The first cited publication by Shirley A. M. Baert, is focused on guide-wire extraction 
based on roug^ estimation using template matching followed by optimization based on 
threadlike structure enhancement, which is not easy to directiy perform in a noisy images. 

30 The second cited publication, by loannis Kompatsiaris et alii, is focused on visibility of the 
stent after stent deployment. 

However, before introduction of the stent, the doctor encounters a first very difficult 
step, which consists in passing the stenosis for the first time with the guide-wire. This is one 
of tiie most di£Qcult tasks the doctor must perform, because the passage the doctor has to 
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Steer through, which is the stenosed part of the artery, is very rough, irregular and limited. 
Besides, the artery walls are mainly radio-transparent, which may necessitate the use of 
contrast agent in order to visualize and localize the stenosed part of the artery. When used, 
contrast agent must be introduced only in very small quantity during very small time duration 
5 because it presents toxicity for the patient. So, using contrast agent provides an improved 
visualization of the artery only during about a dozen of seconds. The doctor must take 
advantage of this short moment to push the guide-wire forward, with the disadvantage that, 
due to the contrast agent, he does not see said guide-wire any more. While the contrast agent 
is operating, the doctor only sees the artery. He anew sees the guide-wire when the contrast 

10 agent has been eliminated through the patient's vessel system, after a few moments. 

Indeed, this step of introducing the guide-wire for the &st time is a very difficult step, 
due to the bad condition of imaging. The doctor can only follow the intervention on a screen, 
since in this intervention the rib cage of the patient is not open. The guide-wire for the 
angioplasty is generally introduced through an opening in the femoral artery and pushed 

15 through the arteries until the aortic cross. Beyond the artery cross, the guide-wire is 

introduced into the coronary tiiat has been diagnosed as suffering of a stenosis. Not only are 
the guide-wire and the stenosed artery walls observed in noisy fluoroscopic images, but also 
they show low radiogr^hic contrast that makes evaluation of the guide wire motion very 
difficult. Besides, during this first phase of the angioplasty intervention, the guide-wire is 

20 moving with respect to the artery, the artery is moving xmder the influence of the cardiac 

pulses, and said artery is seen on a background fliat is moving under the influence of patient's 
breathing and heart beats. These movements make following the guide-wire introduction 
under fluoroscopic imaging still more difficult to perform. Moreover, these movements make 
zoomiug inefficient because the object of interest may get out of the zoomed image firame. 

25 The method of the first cited publication by Shirley A. M. Baert is of no help for 

localizing the guide-wire with respect to a stenosed part of the artery during the step of first 
introduction through said stenosed part, since this method estimates the position of the guide 
in world coordinates and does not provide means to suppress the drawback of the artery 
motion in the images. The method of the second publication by loannis Kompatsiaris et aliU 

30 is of no help for localizing the guide-wire during the step of first introduction through the 
stenosed part of the artery since this method is only focused on visibility of the stent after 
stent deployment. 

Haice, at the difference witii tiie cited prior arts, it is an object of the present 
invention to propose an imaging system having means to improve the visualization and 
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localization of the guide wire with respect to the artery walls, in real time, based on the 
detection of the tip of the guide-wire and the registration of the images. This improved means 
may be used for instance during the fibrst interventional step of guide wire introduction, when 
the guide wire is passed tiurough the stenosed part of the artery for the first time. This first 
5 interventional step of guide wire introduction is very important, because if the interventional 
step that consists in introducing this guide-wire for tihie first time fails, then the following 
steps of stent positioning and deployment are no possible. The alternative to the angioplasty 
is a heavy intervention for bridging the stenosed part of the artery. This heavy intervention 
comprises opening the rib cage of the patient, which leaves much more sequels than the 
1 0 angioplasty intervention. 

According to the invention, these problems are solved by an imaging system having 
means to process guide-wire tip information for determining the position of all the points of 
the skeleton of the guide-wire tip, and means to fiirther register the region of interest around 
said position and means to enhance the visibility of the artery walls and guide wire in said 
1 5 region of interest. This system has means to display in real time the processed images during 
tihie angioplasty intervention for the doctor to correctly follow this intervention. This viewing 
S3^tem has processing means comprising tip detection means, registering means, enhancing 
means and zooming means that permit of accurately visualizing the guide wire with respect 
to the stenosed artery walls. It is also an object of the invention to provide such a system 
20 whose image processing means is user-actuatable. 

Such a system is claimed in Claim 1 and in dependent Claims. An image processing 
method to be used in the system, a program product to implement the steps of this method 
and an examination apparatus for helping visualization of interventions having such a system 
are further claimed. 
25 List of Drawings 

Embodiments of the invention are described hereafter in detail in reference to 
diagrammatic figures wherein: 

FIG.l is a functional block diagram of the registration means of the imaging system; 
FIG.2A illustrates the guide-wire tip and FIG.2B the skeletonized guide-wire tip; 
30 FIG.3 is a fimctional block diagram of the enhancement means of liie imaging system; 

FIG.4 is a functional block diagram of the general means of the imaging system; 
FIG.5A, 5B, 5C illustrate the steps of determining motion vectors in a ROI; 
FIG.6A and 6B illustrate the step of introduction of the guide-wire tip in an arterj^ 
FIG.7 illustrates the step of overlaying live images and peri-intervention images; 
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enhancing and zooming structures in noisy images. The viewing system and the image 
processing method of the invention are described hereafter in an example of application to 
the medical field of cardiology. In said application, the object of interest is an artery, which is 
observed during a medical intervention called angioplasty, in a sequence of X-ray 

10 fluoroscopic images called angiograms. 

Referring to FIG.6A, the angioplasty is a medical intervention that usually comprises 
several stages for enlarging an artery at the location of a lesion called stenosis. In a 
preliminary stage, the practitioner localizes a stenosis 3 in a patient's artery 4 as best as 
possible in peri-interventional images. These unages are obtained while using a contrast 

1 5 agent introduced in the artery in order to improve the image contrast and witii a conventional 
dose of x-rays. The guide-wire may be abready present, but has not yet been passed through 
the stenosed part of the artery. Then, the medical intervention is performed while imaged on 
a screen under fluoroscopy, i.e. with small doses of x-rays. In order to minimize the time 
duration of the intervention, and the total dose of x-rays to the patient, the intervention must 

20 be performed as quickly as possible. Hence, the images must be improved. 

After passing a thin guide-wire through arteries imtil the aortic cross and then towards 
the stenosed artery 4, using a catheter 9, the angioplasty includes a stage of introduction of 
said thin guide-wire 1 through the small lumen of the artery portion 3 at the location of the 
stenosis. 

25 After introduction of the guide-wire, the angioplasty comprises fiirther steps, which 

are not illustrated, of: 

introduction of a monorail, which is guided by said guide-wire, which has a 
first balloon wrapped around its extremity, without stent; and positioning said first balloon in 
the artery portion 3 at the stenosis location; 
30 inflation of this first balloon for expanding the narrow lumen 3 of the artery 4 

at the location of Ihe stenosis; then, removal of the first balloon with the first monorail; 

again using the thin guide-wire 1 for the introduction of a second monorail 
with a second balloon wrapped around its extremity, and with a stent around said second 
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balloon; and positioning said second balloon with the stent at the location of the stenosis in 
the previously eiqpanded lumen of the artery 4; 

inflation of the second balloon in order to expand &e coil forming the stent, 
which becomes the expanded stent embedded in the artery wall; and 
S considering the expanded stent as a permanent implant, removing the second 

balloon attached to the second monorail, the guide-wire 1 and catheter 9. 

The medical intervention called angioplasty is difficult to carry out. The difficulties 
arise because of badly contrasted medical images, where the guide-wire, balloon, stent and 
vessel walls are hardly distinguishable on a noisy background due to the feeble dose of x-rays 

10 in fluoroscopy, and are moreover submitted to motions due to the patient's breathing and 
heart beatings. Some phases of this intervention are very critical. Particularly, the step of 
passing the guide-wire tip through the artery lumen at the location of the stenosis is very 
difficult for the doctor and critical for the patient. This is also a key step in angioplasty, since 
the other steps of angioplasty are not possible without said first step already performed. 

15 As illustrated by FIG.2A, the guide-wire tip 2 is a small metalUc feature disposed at 

the extremity of the guide-wire 1. The guide-wire tip is not rigid, but is bendable in order to 
follow the shape of the artery. This guide-wire tip is about 1cm to several cm long and is 
tiucker than the guide-wire. The guide-wire tip is very radio-opaque, compared to the artery 
walls and the guide-wire itself that are hardly distinguishable in the fluoroscopic images and 

20 that are difficult to visualize when moving the guide wire forwards inside the artery. 

Hence, according to the invention, it has been chosen to use the radio-opaque 
property of the guide-wire tip in order to localize this feature in a Region Of Interest ROI of 
the images. The system of the invention has means to detect the guide wire tip, to determine 
the position of all the points of the skeleton of the guide-wire tip, defined for example by co- 

25 ordinates x, y, as illustrated by FIG.6B, in order to determine a field of motion vectors that is 
used to register the images with respect to a reference image. The system has further means 
to enhance and zoom the threadlike structures formed by the guide-wire and the artery walls 
in the images. In the resulting sequence of images, the contrast of the guide-wire and artery 
walls is improved, the motion is corrected, zooming may be used to enlarge these structures 

30 and the following of the intervention can be substantially continuous. Enhancing the features 
is of importance for the doctor, since at titie beginning it is hardly possible to distinguish 
anything in the acquired images so that pushing the guide-wire forwards without visibility 
can disrupt the artery and arm the patient. Registering the images in real time is also 
particularly important for zooming. The images can be fiulher improved by superimposition 
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of Static peri-interventional views and live views. The system of the invention has means to 
perfomi all these operations. 

The user is the actor of the medical intervention and can have the possibility to 
intervene at each moment. First of all, the user might choose a Region Of Literest ROI in the 

S images. Besides, the user has at his disposal control means 58, shown in FIG.4 and FIG.8, to 
activate and control the image processing means. These control means 58 comprises starting 
means, for the user to start the processing operation, to control the duration of the processing 
operation and to end the processing operation. The processing means 10, 20, denoted by 
stenosis boosting means, is actuated by the user control 58 during one of the above-cited 

10 phases. 

Referring to FIG.l, tiie system of the invention has first image processing means 10 
for providing a field of motion vectors for estimating the motion of the guide-wire tip. The 
motion estimation is performed in a first Region Of Interest ROI. The means 10 includes a tip 
detector 1 1 that solves the problems of automatically detecting the guide-wire tip in the 

15 images of the sequence displayed on the screen during the angioplasty. The tip detector 
provides a skeleton denoted by SKT formed of a chain of pixels. The tip detector 1 1 may 
include ridge filter for detecting ridge corresponding to the tip, threshold means to distinguish 
the tip &om other features, pixel gathering means and comiectivity analysis means to finally 
form a chain of pixels as illustrated by FIG.2B. This tip detection is performed in all the 

20 images of the sequence. The skeleton is denoted by So in a first image of the sequence to be 
considered, and is denoted by St at a fiirther instant t. The skeleton detection permits of 
estimating the position of all the points of said skeleton of the guide wire tip in a referential 
of the images. 

An extension of the tip skeleton So-St in the succeeding images of the sequence is 
25 then performed by the skeleton extension means 12. This skeleton extension is performed 
spatially in each image separately. The spatial extension may be performed linearly or 
according to a curve, in the forward and/or the backward directions. The skeleton may be 
spatially extended of a length D as shown in FIG.6B. Information data of the extended 
skeleton position are denoted by S VS't in tiie ROI. Information data of the first extended 
30 skeleton position S'o is memorized in memory means MEM. 

As illustrated by FIG.5A, 5B, using the information data of the first extended skeleton 
position S'o stored in MEM, skeleton vector field estimation means 13 compares the extended 
skeleton position S'o at a first instant 1r=0, in the referential of a first image called reference 
image, and in the ROI, with the extended skeleton positions S't at succeeding instants t. This 
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permits of estimating, as illustrated by FIG.IC, a field of motion vectors V, which makes the 
extended skeleton positions S\ correspond to the extended skeleton position So- Then a 
vector field extension is performed by the vector field estimation means 13, for instance 
using a known technique of warping. Using this technique, a geometric transform is created. 

5 Then, said geometric transform is applied for perfonrdng a registration operation in the ROL 
Referring to FIG.3, the system of the invention further has second means 20 for 
enhancing threadlike features in the ROI in the image sequence. The means 20 comprises 
registration means 21 that receives the motion vectors V and the pixel intensity of the 
original image I(t) and that registers the guide-wire tip in the different images at instants t 

10 with respect to the guide wire tip at the instant t=0 considered the referential of the reference 
image. Since the guide-wire tip moves together with the artery, said artery is also registered 
with respect to the referential of the reference image and does move any longer from one 
image to a succeeding image of the sequence. Then, the means 20 comprises ridge 
enhancement means 22 for enhancing the threadlike structures in the ROI. Hence, the 

15 threadlike structures constituted by the guide-wire and the artery walls are enhanced in the 
ROL Then temporal integration means 23 performs a temporal integration. Said temporal 
integration can be achieved by an averaging technique of the pixel intensity of the registered 
threadlike structures over several images for instance while tiie background that is still 
moving is blurred. The temporal integration can be achieved by any other known technique. 

20 The means 20 further comprises dynamic adaptation means 24 for providing a resulting 
image R(t) which may be zoomed according to the need or desire of the doctor. 

Now, the doctor has means to visualize the artery walls that are no more moving from 
one image to a succeeding image, that are no more invisible but instead that are enhanced. 
Referring to FIG.4, the doctor has the means 58 to start the processing operation called 

25 Stenosis Boosting 10, 20. When started, the images pass from I(t) to Stenosis Boosting 10, 
20. The data that are estimated for registering the ROI ia the images, such as S*o, are 
memorized in MEM, and used for each image registration. From the means 10, the position 
x,y of all the pixels of the skeleton of the guide-wire tip are estimated. Thus, the doctor can 
push slowly the guide forward in the stenosed part of the artery, which is registered and 

30 enhanced on a blurred backgroimd. The start means 58 can start an automatic device 33 for 
delivering a small amount of contrast agent to be introduced in the artery at appropriate 
instant witihi respect to the instants when the doctor push the guide forward while observing 
the processed images R(t). The contrast ag^t is introduced by appropriate means (catheter). 
In particular, a great advantage of the invention is that it is no more necessary to inject 
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concentrated contrast agent in the artery of the patient Since the artery walls are enhanced, 
the doctor can see sufficiently the position of the artery wall to motion the guide-wire 
properly and safely. However, if the doctor wants to use a contrast agent, it is appropriate to 
use diluted contrast agent instead of concentrated contrast agent, which permits of visualizing 
S the guide-wire tip while the diluted contrast agent is perfusing, which is not possible with 
concentrated contrast agent. Besides, diluted contrast agent is less toxic for the patient than 
concentrated contrast agent. Using diluted contrast agent permits to use it on a longer laps of 
time without inconveni^t for the patient, thus a few minutes instead of a dozen of seconds 
with a concentrated contrast agent. The doctor may operate the start means as often as 

10 necessary, imtil the guide-wire tip is safely passed through the stenosed part of the artery. 

In order to fiirther improve the accmacy of the images to be displayed by the display 
means for visualization by the doctor, a ROI can be superimposed to a corresponding ROI of 
a peri-interventional image. Referring to FIG.7, this operation can be performed using a 
block-matching technique. The data of the live image R(t) are entered together with data of a 

15 sequence of peri-interventional images denoted by Jo to !„ in a gray level based registration 
means 41. The gray level based registration means 41 can provide the current position of the 
guide-wire tip as shown in the live image R(t) superimposed on the features of the sequence 
Jo-Jn of peri-interventional images. The resulting combined features form a sequence of peri- 
interventional images Po-Pn denoted by OVERLAY 1 that are enhanced by the features of 

20 R(t). Or, using an a-blending technique, the gray level based registration means 41 can 
provide a live image denoted by R'(t), which is an overlay, denoted by OVERLAY 2, of a 
peri-interventional image Ji among the set Jo-Jn, which is determined by the means 41, 
superimposed on the live image R(t). 

Hence the system of the invention is able to provide Uve images that are registered 

25 with respect to one another, resulting in live enhanced images. This system is also able to 
provide live images that are registered with respect to peri-interventional images. 

In a preliminary step, the peri-interventional images can be registered one with 
respect to another in a referential formed by patient's breathing characteristics and ECG 
points. These peri-interventional images are then seen as a cloud of points in this referential. 

30 Hence, an image I(t) would not be easily matched to one image of this set of peri- 
interventional images. Now, according to the invention, it is not the image I(t) that is 
proposed to be matched to one image of the set of peri-interventional images, but instead it is 
the registered and enhanced image R(t). Hence, a robust registration between the live and the 
peri-interventional images can be obtained. 
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FIG.8 shows a diagram of a medical examination apparatus 50. The apparatus has 
means 51 for acquiring digital image data of a sequence of images, and is coupled to a 
medical viewing system S3 as described above, for processing these data according to the 
processing method cited above. The medical viewing system is generally used in the 
5 intervention room or near the intervention room for processing real time images. Steps of the 
present method can be ^plied on stored peri-interventional images, for example for 
estimating medical parameters. The medical examination sqpparatus provides the image data 
by connection 57 to the system 53. The system provides processed image data to display 
means and/or storage means. The display means 54 may be a screen. The storage means may 

10 be a memory of the system 53. Said storage means may be alternately external storage 

means. This image viewing system 53 may comprise a suitably programmed computer, or a 
special purpose processor having circuit means such as LUTs, Memories, Filters, Logic 
Operators, that are arranged to perform the functions of the method steps according to the 
invention. The system 53 may also comprise a keyboard 55 and a mouse 56. Icones maybe 

1 5 provided on the screen to be activated by mouse-clicks, or special pushbuttons may be 
provided on the system, to constitute control means 58 for the user to start, to control the 
duration or to stop the processing means of the system at chosen stages or phases. 



